Skip to content

Support target database in write-back sink#18058

Merged
jt2594838 merged 7 commits into
apache:masterfrom
Caideyipi:writeback-sink-target-database
Jul 1, 2026
Merged

Support target database in write-back sink#18058
jt2594838 merged 7 commits into
apache:masterfrom
Caideyipi:writeback-sink-target-database

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Changes

  • Add sink.database / connector.database support for write-back sink data writes.
  • Validate write-back sink database names on the sink side instead of relying on generic path validation.
  • Use table-model create-database validation for table-model database names.
  • Use tree-model create-database validation semantics for tree-model database names, including root. requirement, legal tree path parsing, database-name pattern check, and length limit.
  • Support multi-level tree-model target databases such as root.target.db.
  • Reject table-model writes when the event or target mapping produces an invalid table-model database, for example tree database root.a.b mapping to table database a.b.
  • Rewrite target database for both table-model and tree-model insertion statements.
  • Add UT coverage for target database validation boundaries, target database customization/mapping, raw tablet event rejection, and statement event rejection.
  • Update tree-model write-back sink IT coverage to use a multi-level target database.

Semantics

  • sink.database=target is treated as a table-model database and mapped to tree database root.target for tree writes.
  • sink.database=root.target and sink.database=root.target.db are treated as tree-model databases.
  • A multi-level tree-model database such as root.target.db is valid for tree-model writes.
  • The corresponding table-model database name for root.target.db is target.db, which is invalid in table model and must reject table-model events.
  • Table-model events always validate their event database name with table-model create-database rules before applying a configured target database, so source values like root.a.b / a.b are rejected.
  • Tree-model events always validate their event database name with tree-model create-database rules before applying a configured target database.

Validation

  • mvn spotless:apply -pl iotdb-core/datanode
  • mvn spotless:apply -pl integration-test -P with-integration-tests
  • git diff --check
  • mvn -pl iotdb-core/datanode -Dtest=PipeSinkTest test was attempted, but datanode compilation fails before test execution in existing generated Freemarker sources unrelated to this change, including missing IFill, IFillFilter, LinearFill, Accumulator, org.apache.commons.collections4.comparators, and IoTDBConfig#getModeMapSizeThreshold().

Comment on lines +665 to +667
if (!startsWith(deviceNodes, sourceDatabaseNodes)) {
return devicePath;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When will this happen?

@jt2594838 jt2594838 merged commit 1c8ed31 into apache:master Jul 1, 2026
40 checks passed
@jt2594838 jt2594838 deleted the writeback-sink-target-database branch July 1, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants